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This paper describes a transfer-of-training study performed in the NASA Ames Vertical Motion Simulator. 
The purpose of the study was to investigate the effect of false tilt cues on training and transfer of training of 
manual roll control skills. Of specific interest were the skills needed to control unstable roll dynamics of a 
mid-size transport aircraft close to the stall point. Nineteen general aviation pilots trained on a roll control 
task with one of three motion conditions: no motion, roll motion only, or reduced coordinated roll motion. 
All pilots transferred to full coordinated roll motion in the transfer session. A novel multimodal pilot model 
identification technique was successfully applied to characterize how pilots’ use of visual and motion cues 
changed over the course of training and after transfer. Pilots who trained with uncoordinated roll motion had 
significantly higher performance during training and after transfer, even though they experienced the false 
tilt cues. Furthermore, pilot control behavior significantly changed during the two sessions, as indicated by 
increasing visual and motion gains, and decreasing lead time constants. Pilots training without motion showed 
higher learning rates after transfer to the full coordinated roll motion case. 


Nomenclature 


Ad 

disturbance amplitude, deg 

N d 

number of disturbance sine waves 

At 

target amplitude, deg 

N t 

number of target sine waves 

e 

error signal, deg 

n 

pilot remnant, deg 

fd 

disturbance forcing function, deg 

Xf' 

run number 

ft 

target forcing function, deg 

s 

Laplace operator 

9 

standard gravity, ft s -2 

t l 

lead time constant, s 

H c 

aircraft dynamics response 

T m 

measurement time, s 


pilot motion response 

u 

pilot control input, deg 

H 0 l,d 

disturbance open-loop response 

Vs 

simulator lateral displacement command, ft 

H ol , t 

target open-loop response 



H v 

pilot visual response 



Hvms 

VMS motion logic dynamics 

Symbols 


I<m 

K v 

K, 

Ky 

N 0 

Aoo 

pilot motion gain, — 
pilot visual gain, — 
simulator roll motion gain, — 
simulator lateral motion gain, — 
initial value level 
asymptotic level 

Cnm 

Cr 

neuromuscular damping, — 

roll washout filter damping ratio, — 

A 

7~m 

T v 

rate of exponential decay or growth, — 
motion delay, s 
visual delay, s 
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0 

roll angle, deg 

^C,t 

target crossover frequency, rad s _1 

0S 

simulator roll command, deg 

Ud 

disturbance sinusoid frequency, rad s _1 

^Pm,d 

target phase margin, deg 


measurement time base frequency, rad s _1 

m,t 

disturbance phase margin, deg 

k' nm 

neuromuscular peak frequency, rad s _1 

CO 

frequency, rad s _1 

C0 r 

roll washout filter break frequency, rad s _1 

c,d 

disturbance crossover frequency, rad s _1 


target sinusoid frequency, rad s _1 


I. Introduction 

Airline pilots will soon be required to perform full stall recovery training in simulators. However, flight simulators 
currently used for pilot training do not represent aircraft behavior in upset situations that take the aircraft out of its 
normal flight envelope. Post-stall aircraft models need to be implemented to correctly simulate the aircraft response 
after the stall point. In addition, motion cues need to adequately represent this response to ensure the skills learned in 
simulator training are directly usable in real flight . 1 This paper investigates how different motion configurations affect 
training and transfer of training of the manual control skills required to control unstable roll dynamics of a medium- 
size transport aircraft near the stall point. We specifically focus on the false tilt cues that are often encountered in 
current hexapod training simulators. 

Previous research showed that simulator motion significantly affects multimodal pilot control behavior and per- 
formance . 2-5 Motion feedback provides more rapid information than visual feedback on the rate of change of the 
controlled aircraft state, allowing a human operator to increase disturbance-rejection performance. Little research has 
been performed on the effects of motion on training and transfer of training of manual control skills. Levison used 
pilot modeling techniques to characterize how pilot control behavior develops over the course of training and after 
transfer . 6 A novel multimodal pilot model identification technique was successfully applied by Pool et al. to charac- 
terize changes in multimodal pilot control behavior during training . 7 This approach will also be used in the current 
study. 

The paper is structured as follows. First, a brief overview of roll motion cueing considerations is given. Then, 
the paper describes the experimental design, which used the world’s largest flight simulator. After presentation of the 
results, a discussion follows. 


II. Roll Motion Cueing 

In flight simulators, gains and washout filters attenuate the aircraft model rotational and translational accelerations 
to keep the motion platform within its displacement, velocity and acceleration limits. The tuning of the gains and filter 
break frequencies always requires a tradeoff between different degrees of freedom. This is even more so for hexapod 
motion platforms for which the same six actuators are required to move the platform in each degree of freedom. 
In addition, the rotational and translational accelerations in certain degrees of freedom, such as roll and lateral, are 
coupled and need to be treated together in the motion tuning process. First, a certain amount of lateral translational 
motion must accompany a rolling motion to translate the center of rotation properly. Second, roll-tilt coordination 
is often used to simulate sustained lateral accelerations. Finally, for coordinated roll motion, lateral platform motion 
provides an acceleration to counteract the false tilt cue that would result from only rolling the cockpit . 8 - 9 

During a coordinated turn in an aircraft, the centrifugal acceleration will cancel out the tilt component of gravity 
in the body reference frame, keeping the apparent gravity vector g' parallel to the pilot’s spine. In order to accurately 
reproduce this in a simulator, substantial lateral travel is required. However, due to the limited motion space of hexapod 
motion platforms, this compensatory lateral acceleration [g tan(<j))] can only be provided for a very limited amount of 
time (Fig. lb). In the case where no lateral accelerations are provided, as depicted in Fig. lc, the pilot perceives the 
horizontal component of the gravity vector and feels a false tilt cue [g sm(</>)]. 

To reduce the amount of lateral displacement used to eliminate the false tilt cues, different tradeoffs can be made 
between motion tuning parameters in the roll and lateral degrees of freedom. First, the roll angle of the motion 
platform can be reduced relative to the model (and thus the visual scene). This requires less lateral travel, since the 
magnitude of the tilt cue is reduced. Second, the amount of lateral translational motion required for coordination can 
be reduced, allowing for some false cues to occur. Previous research has investigated the tradeoff between these two 
options and looked at the effects on pilot performance and subjective ratings . 9 Another study looked at the effects of 
various degrees of lateral acceleration washout in a roll control task on pilot control behavior and performance . 8 In 
both studies, the effects of false tilt cues on pilot performance were limited, and in the later study the effects on manual 
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(a) No motion. 


(b) Coordinated roll. 


(c) Uncoordinated roll. 





Figure 1. Apparent gravity orientation during a coordinated and uncoordinated roil in a flight simulator. 


control behavior were also limited. However, significant differences were found between pilots’ subjective ratings in 
both studies, with pilots consistently noticing the false tilt cues when the peak spurious lateral specific force was above 
0.1 g. 

Although previous studies did not find many significant differences in pilot performance and control behavior, the 
effects of false tilt cues on pilot training and transfer of training have never been investigated. This is the purpose 
of the current study for which an experiment was performed on the Vertical Motion Simulator (VMS) at NASA 
Ames Research Center. We specifically analyzed the control of unstable roll dynamics of a transport aircraft near the 
stall point. For such unstable dynamics, motion has been shown to significantly affect manual control behavior and 
performance. In order to analyze the effect on transfer of training of manual roll control skills, pilots first trained with 
one of the three motion configurations visualized in Fig. 1 . All these motion configurations were tuned in such a way 
that they could be performed on a medium-size hexapod motion base with 60-inch actuators. For the coordinated roll 
motion condition, this meant that roll accelerations were significantly attenuated to allow for full compensation of the 
lateral tilt cues. The no-motion configuration was performed as a baseline condition. After the training session, each 
pilot performed a transfer session, which utilized the entire lateral track of the VMS for full coordinated roll motion. 
More details on the motion configurations are given in Section III. 

III. Experimental Design 


III.A. Control Task 

Fig. 2 depicts the roll control task performed by pilots in the VMS. The pilots' task was to actively minimize the roll 
error, e, presented on a compensatory display, which resembled a basic Primary Flight Display (PFD). In addition to 
the visual stimuli presented on the display, roll stimuli were provided by the simulator motion logic with dynamics 
Hvms- The level of motion feedback pilots received varied according to the motion configurations presented in Fig. 1. 
Pilots acted on the controlled roll dynamics, H c , by making control inputs, u, with a sidestick. 



Figure 2. Control Task. 
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In the control task of Fig. 2, a human operator can be modeled by linear response functions. 10 A remnant signal, n, 
accounts for measurement noise and nonlinearities not captured by the response functions. In order to separate pilots’ 
visual and motion responses, H v and H m , two independent sum-of-sines forcing functions were used in the control 
task. 2 A disturbance forcing function, fd, was added to the controlled element output, feeding directly into the display 
and VMS motion system. A target forcing function, f t , feeds into the display only through the error, e, the difference 
between f t and the disturbed roll attitude, </>. The disturbance forcing function had a power four times greater than the 
target forcing function, making the control task primarily a disturbance-rejection task. 


III. A. 1 . Controlled Aircraft Dynamics 

Pilots controlled the dynamics of a mid-size twin-engine commercial transport aircraft with a gross weight of 185,800 
lbs. The aircraft model was linearized close to the stall point, at an altitude of 41,000 ft and an airspeed of 150 kts. At 
this flight condition, the roll dynamics of the aircraft are unstable, and defined by the following transfer function: 


H c (s) = 0.78208 


(s 2 + 0.2175s + 0.5861) 

(s + 0.7599) (s - 0.02004) (s 2 + 0.1133 + 0.6375) 


( 1 ) 


where s represents the Laplace operator. Figure 3 depicts the frequency response of this transfer function. At low 
frequencies, < 0.75 rad/s, the dynamics approximate a single integrator (-), whereas at frequencies above 1.0 rad/s 
they approximate a double integrator (- 2 ). 


III. A. 2. Forcing Functions 

The target and disturbance forcing functions are sum-of-sines signals defined by: 


N d , t 

fd,t(t) = A d ,t(k)sin[w d ,t(k)t + (j)d,t(k)\ ( 2 ) 

k = l 

where A(k), u>(k), and cf>(k), respectively, indicate the amplitude, frequency, and phase of the k th sine in fd or f t . Nd,t 
represents the number of sine waves. Subscripts d and t are the disturbance and target forcing function, respectively. 
Both fd and f t consist of ten individual sinusoids with a different amplitude, frequency, and phase. A summary of all 
forcing function properties can be found in Table 1 . 

The frequencies of the sinusoids, utd{k) and -o f {k), were all integer multiples of the measurement time base fre- 
quency, uj m = 27r/T m = 0.0767 rad/s, where T m = 81.92 s is the measurement time. The selected integer multiples 
were used in a number of earlier experiments and ensured that the ten sinusoid frequencies in each signal covered the 
frequency range of human control at regular intervals on a logarithmic scale. 

A second-order low-pass filter was used to determine the amplitudes of the individual sines for both the target 
and the disturbance forcing functions. 3 This second-order filter reduces the magnitude of the amplitudes at the higher 
frequencies, yielding a tracking task that is not overly difficult. The amplitude distributions Ad(k) and A t (k) were 
scaled to attain variances for fd and f t of 2.0 deg 2 and 0.5 deg 2 , respectively; meaning the target had only 25% of the 




l j, rad/s lj, rad/s 

Figure 3. Frequency response dynamics. 
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Table 1. Forcing function properties. 



target, f t 



disturbance, fd 


n t , - 

ujt , rad s 1 

A t , deg 

4>t , rad 

rid, ~ 

ujd, rad s 1 

Ad, deg 

4>d, rad 

6 

0.460194 

0.7809 

-1.4796 

5 

0.383495 

1.5021 

-0.7257 

13 

0.997088 

0.5461 

-0.0745 

11 

0.843689 

1.1355 

-0.9253 

27 

2.070874 

0.2464 

0.7006 

23 

1.764078 

0.5656 

0.6432 

41 

3.144661 

0.1327 

-1.9563 

37 

2.837864 

0.2881 

2.7264 

53 

4.065049 

0.0891 

-2.8131 

51 

3.911651 

0.1753 

-0.1488 

73 

5.59903 

0.0552 

2.1026 

71 

5.445632 

0.1065 

1.8202 

103 

7.900001 

0.0351 

-2.6178 

101 

7.746603 

0.0666 

-2.0536 

139 

10.66117 

0.0256 

2.255 

137 

10.50777 

0.0482 

1.9364 

174 

13.34563 

0.0214 

-0.6739 

171 

13.11554 

0.0402 

1.0396 

229 

17.56408 

0.0182 

0.1942 

226 

17.33398 

0.034 

-2.6909 


power of the disturbance. With a small enough variance for the target signal, the pilot will not be consciously aware of 
the visual/motion discrepancies and will operate as if there were only the disturbance signal. However, if the variance 
is too small, signal-to-noise problems will arise. This relative strength of the target and disturbance signals, and the 
amplitude variation determined by the second-order filter were successfully applied to measure pilot tracking behavior 
in earlier experiments. 3 ' 5 

To determine the forcing function phase distributions, numerous random sets of phases were generated. Two 
sets of phases were chosen for the target and disturbance forcing functions that yielded signals with a Gaussian-like 
distribution and an average crest factor. 1 1 

III.B. Apparatus 

This experiment used the VMS with the rotorcraft cab (R-CAB), see Fig. 4. 12 A summary of VMS capabilities can 
be found in Ref. 13. Pilots were seated on a chair positioned in the center of the cab and controlled the roll attitude 
through a joystick located on the right side of the seat as seen in Fig. 5. Pilots had a simplified PFD in front of them 
that displayed the roll error (see Fig. 6). The out-the-window view consisted of a green ground plane and monotone 
blue sky. No other visual features were present in the out-the-window view. 

The collimated out-the-window display of R-CAB consists of three cathode ray tube monitors casting a high- 
quality image on three spherical mirrors. The three windows span a field of view from +78° to -77° in azimuth and 
+12° to -17° in elevation. The visual system equivalent time delay was 62 ms, which is similar to the time delays of 
the motion system. 13 

III.C. Experimental Conditions 

For each pilot, the experiment consisted of a training and a transfer session. The motion configuration used during 
the training session was the only independent variable of the experiment. Three different motion configurations were 
used for training: no motion, roll motion only, and coordinated roll motion (see Section II). All these motion training 
conditions were tuned in such a way that the motion could be executed on a medium-size hexapod motion base. Full 





Figure 4. Vertical Motion Simulator. 


Figure 5. Cockpit setup. 


Figure 6. Primary flight display. 
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coordinated roll motion was used in the transfer session after each pilot trained on one of the aforementioned training 
conditions. Table 2 summarizes the training motion conditions. 

The motion logic used in the experiment is depicted in Fig. 7. I\ r is the roll motion gain, and Q r and ui r are the 
damping ratio and break frequency of the roll washout filter, respectively. K y is a gain on the lateral accelerations. 
Note that Fig. 7 only provides an overview of the motion logic most relevant to this experiment. The full motion logic 
contained algorithms to transform linear and rotational accelerations from the body to inertial reference frame and 
to translate the center of rotation. The center of rotation was located 3.75 ft below the pilot eye point in all motion 
configurations. Table 2 provides the values of the motion logic parameters for each motion condition. Note that the 
roll washout was also used for the roll-motion-only and full-coordinated-roll-motion configurations, but with a very 
low break frequency. 


Table 2. Training motion groups. 


Group 

Label 

Training Motion (K r ,^ r ,uj r ,K y ) 

Transfer Motion (K r 

,C r ,Ul r ,K y ) 

1 

NM 

no motion 

(0.0, , -) 

full coordinated roll motion 

(1.0, 0.707, 0.2, 1.0) 

2 

RO 

roll motion only 

(1.0, 0.707, 0.2, 0.0) 

full coordinated roll motion 

(1.0, 0.707, 0.2, 1.0) 

3 

CR 

hexapod coordinated roll motion 

(0.5,0.707, 0.8, 1.0) 

full coordinated roll motion 

(1.0, 0.707, 0.2, 1.0) 


aircraft model roll simulator roll 

accelerations and lateral accelerations 



Figure 7. Simulator motion logic. 


III.D. Participants and Procedures 

Nineteen general aviation pilots participated. Three pilots had 1,500 or more hours of flying, with the highest number 
of hours at 2,000. The average number of flight hours was 534 with a standard deviation of ± 654. Most pilots had 
fixed-base simulator experience, with an average of 213 hours and standard deviation of ± 302. Pilots fell between 
the ages of 20 and 70, with the majority under 40 years with an average age of 28.8 years and a standard deviation of 
± 12.6 years. 

Before the start of the experiment, each pilot received a thorough briefing informing them of the purpose of the 
experiment and overall procedures. In addition, the briefing explained the goal of the task and suggestions on how to 
compensate for the disturbances more effectively. Pilots were not given any details about the motion conditions. To 
motivate pilots to improve their performance, they were told the current best score and encouraged to improve on it. 
The pilots were also briefed on good control strategies and given a few examples on paper of good and bad control 
inputs. Finally, pilots were instructed to focus on the PFD (heads down) and not use the out-the-window view. 

For the experiment itself, each pilot came in for two sessions spread over two days, in the morning or afternoon. For 
the first session, the pilot performed training for the task with one of three training conditions mentioned above. Each 
pilot was assigned to a motion group randomly. The next day, the pilot performed the task with the full coordinated 
roll motion condition. Each pilot performed 80, 90-second runs per session, in four sets of 20 with a ten-minute break 
between sets, for a total of 160 runs over the two sessions. During each session, the pilots were allowed to take longer 
breaks if they requested. 

At the end of each run, the root mean square (RMS) of the roll error appeared at the bottom right of the PFD to 
give pilots feedback on their performance. The smaller their score, the better performance they achieved throughout 
that run. 
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III.E. Pilot Control Behavior and Performance Measures 


Several dependent measures were calculated for every pilot to evaluate how control behavior and performance changed 
during training and after transfer. Pilot control intensity and performance were evaluated in terms of the root mean 
square of the lateral control input, RMS U , and roll error, RMS e , respectively. These two measures were calculated 
for every single run. 

In order to characterize pilot control behavior for each pilot, linear transfer functions are used to describe the visual 
and motion responses. Nonlinear behavior and measurement noise are captured by the remnant signal n (Fig. 2). 
According to the crossover model theorem, 10 a human operator adjusts his or her control behavior to the controlled 
dynamics such that the pilot- vehicle open-loop response approximates a single integrator near the crossover frequency. 
The controlled dynamics in Eq. (1) approximate a double integrator in the frequency range where the pilot- vehicle 
crossover frequency is expected for compensatory tracking (1-5 rad/s). This means pilots need to generate lead by 
using visual and, when available, motion cues. 

Taking into account the need for visual lead generation, the visual response transfer function used to model manual 
control behavior in Fig. 2 is defined by: 


H v (s) 


K v (T l s + l)e~ TvS 


LU 


2 

nm 




■ 


and the motion response transfer function by: 


(3) 


Hm(s) = sK rn e~ TmS ™ , 2 (4) 

In this multimodal pilot model representation, pilot equalization is defined by the visual gain K v , the visual lead 
time constant Tj,, and the motion gain K m . The pilot’s limitations are characterized by the visual delay t v , the motion 
delay r m , and the neuromuscular dynamics with a neuromuscular frequency w nm and a neuromuscular damping C, nm . 

It must be noted that H m represents the sum of multiple separate motion feedback channels defining the use of 
angular accelerations detected by the semicircular canals, linear accelerations detected by the otoliths, and motion 
cues from the somatosensory system. This modeling approach has successfully been used in previous research. 14 
Furthermore, except for the VMS motion system dynamics Hy^g, all other simulator dynamics, such as latencies, 
are included in //,, and H m . 

The pilot model defined in Eqs. 3 and 4 contains seven parameters ( K v , Tg, t v , K rn , r m , C, nrn , and uj nm ), which 
were estimated from experiment data using maximum likelihood estimation (MLE). Data were averaged over four 
measurement runs before proceeding with the parameter estimation in order to reduce the influence of the remnant 
on accuracy of the parameter estimates. With 80 runs, this means that control behavior was evaluated 20 times in the 
training and transfer sessions for each pilot. First, a genetic algorithm optimization was used to obtain rough initial 
estimates of the parameters. Following, a Gauss-Newton optimization algorithm obtained the set of parameters that 
provides the pilot model that best fit the experiment data. 15 The control signal variance accounted for (VAF) was 
calculated as a measure of the accuracy of the pilot model in describing the measured control signal data. 

In the frequency domain, pilot performance in attenuating the target and disturbance signals is determined by the 
crossover frequencies and phase margins of the target and disturbance open- loop dynamics, respectively. 16 Using the 
control task diagram in Fig. 2 and the pilot response functions in Eqs. 3 and 4, the disturbance and target open-loop 
responses are given by: 


H 0 i,d (s) = [H v (s) + H V ms (s) H m (s)] H c (a) (5) 

rr M _ H V (S)H C (S ) 

° l ’ t[) 1 + H VM s(s)H m (s)H c (s) W 

The disturbance and target crossover frequencies, l u C) d and w C|t , are the frequencies where the magnitude of the 
disturbance and target open-loop responses is 1.0. The corresponding phase margins, (p rn .d and are the phase 
differences from —180 degrees at these crossover frequencies. 

For each pilot, an exponential decay function was fit on all the dependent measures described above to characterize 
each measure’s change during training or transfer sessions: 

N{ir) = {No - N^e-^ + Woo (7) 
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where iV (l is the initial value of the measure at the start of the session, is the asymptotic value at the end of a 
session, A is the rate of exponential decay or growth, and i r is the run number. 

IV. Results 

This section provides the results of 14 of the 19 pilots who participated in the experiment. After a thorough 
analysis, five pilots were eliminated from the data analysis for the following reasons: one pilot did not finish the 
experiment, two performed below standards, and two produced control inputs that did not allow for accurate pilot 
model parameter estimates. This left four pilots in groups NM and CR, and six pilots in group RO. 

As described in Section III.E, an exponential decay function was fit to each of the dependent measures for both the 
training and transfer sessions of the experiment. An example of a curve fit applied to the performance of one pilot in 
the training and transfer sessions is depicted in Fig. 8. This curve fitting method is particularly useful since it provides 
a measure of the initial performance level No, the theoretical asymptotic performance reached at the end of the session 
A’oc, and the learning rate A. These parameters can be used to analyze the differences in training and transfer of 
training from the different training motion configurations. An analysis of variance (ANOVA) was performed on each 
dependent measure to determine whether differences found between motion groups were statistically significant. A 
summary of the ANOVA results can be found in the appendix. 


(a) Training. 


(b) Transfer. 
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Figure 8. Example curve fit. 


IV.A. Performance and Control Activity 

Pilot performance is defined as the RMS of the roll error presented on the PFD. The lower the value of RMS e , the 
better the performance. Fig. 9a presents the evolution of performance during training and transfer. The curves depicted 
in Fig. 9a are constructed using the curve fitting parameters averaged over all pilots of a motion group. The means and 
95% confidence intervals of the initial and asymptotic performance levels for each group are depicted at the start and 
end of each curve. The confidence intervals are corrected for small sample size. 

In the training phase, the roll-only motion condition yields the best performance. Furthermore, the error bars are 
the smallest out of all conditions, indicating that in the roll-only condition, pilot performance varies less. The variance 
in pilot performance appears to decrease overall in the transfer condition compared to training. Performance in the 
coordinated-roll-training-motion condition (CR) is very similar to performance in the no-motion condition (NM). Note 
that the order of performance levels between motion groups seems to be maintained in the transfer session. In addition. 
Fig. 9a suggests that pilots in the NM and CR motion groups took longer to converge to asymptotic performance in 
the transfer session. 

Fig. 9b depicts the RMS of pilots’ control input. Pilots in the no-motion group seem to control with the lowest 
intensity during training, and have the smallest variance between pilots. In the transfer session, the RO motion group 
has the smallest variance between pilots. This suggests that all pilots who trained with roll motion only control 
with similar intensity in the fully coordinated roll motion condition. Pilots who trained with no motion or reduced 
coordinated roll motion showed much larger variations in control intensity in the transfer session. 
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(a) Roll error RMS. 


(b) Stick deflection RMS. 




run 


run 


Figure 9. Pilot performance and control activity. 


IV.B. Variance Accounted For 

The VAF indicates how much of the variance present in the mea- 
sured signal can be accounted for by the linear pilot model de- 
scribed in Section III.E. Fig. 10 depicts the evolution of the VAF 
for all three training motion groups in the training and transfer 
sessions. Again, the means and 95% confidence intervals of the 
initial and asymptotic values are depicted at both ends of the 
exponential curves. The figure suggests that the VAF at the start 
of the training session is similar for all three motion groups. For 
the RO and CR motion groups, the VAF level has an increasing 
trend during the training session, which suggests that the pilots 
adopt a more linear and constant control strategy while training. 

At the end of the training session, the highest VAF is achieved 
for the CR motion group, whereas the VAF for the NM group is 
the lowest. This result indicates that pilots adopt a more linear control strategy when training with reduced coordinated 
roll motion. The relative order of the VAF is maintained in the transfer session, where full coordinated roll motion 
was present. Fig. 10 also shows that the variance between pilots becomes smaller at the end of training compared to 
the variance at the beginning. 

IV.C. Pilot Model Parameters. 

The estimated model parameters from the pilot model presented in Section III.E are depicted in Fig. 11, which shows 
the evolution of all the parameters in the training and transfer sessions. The curves are obtained using the mean curve 
parameters for each motion group. The error bars indicate the 95% confidence intervals, corrected for small sample 
size, for the initial and asymptotic levels of each curve. 

The visual gain of the pilot is depicted in Fig. 11a. From this figure one can see that the group who trained without 
motion has the lowest visual gain during training. When motion is present during training, the visual gain is higher, 
suggesting that pilots are more confident to act on visual information when performing the task. 

Fig. 1 lb depicts the lead time constant evolution for each of the motion groups. During training, the group learning 
without motion yields the largest value for this parameter, indicating that this group generates the most lead information 
from the visual stimuli. This is expected, since the visual stimuli are the only stimuli pilots can use to obtain lead 
information. While the roll-only motion condition had the largest miscues, the group training with this motion had the 
lowest lead time constant, indicating that pilots generate the least visual lead. The value of the lead time constant for 
the coordinated-roll training group lies between the values for the two other groups. Furthermore, the relative order of 
the lead time constant values between the different groups in the training session is the opposite of what was observed 
for the visual gain. At the end of the transfer session, all three levels of the lead time constant seem to converge to 
the same value. However, the NM group starts with the highest lead time constant in the transfer session and has the 
highest learning rate. 



run 

Figure 10. Variance accounted for. 
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(a) Visual gain. 


(b) Lead time constant. 




(c) Visual delay. 



run 


(d) Motion gain. 


(e) Motion delay. 




(f) Neuromuscular damping. 


(g) Neuromuscular frequency. 
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Figure 11. Pilot model parameters. 

10 of 15 


American Institute of Aeronautics and Astronautics 


(a) Pilot visual frequency response. 


(b) Pilot motion frequency response. 



cj, rad/s 

Figure 12. Mean pilot model frequency responses after training. 


a rad/s 


The evolution of the pilots’ motion gain can be seen in Fig. 1 Id. Note that this parameter is estimated only when 
motion stimuli were present. During the training session, the motion gain shows an increasing trend. This suggests 
that pilots learn to act more confidently on the motion stimuli over the course of training. The roll-only group had the 
highest motion gain. In the transfer condition, no significant differences can be observed between the motion groups. 
However, pilots who trained without motion seem to have a lower motion gain at the beginning of the transfer session, 
which then increases during the session to the asymptotic level of the groups who trained with motion. 

The remaining parameters presented in Fig. 11, all related to physical limitations of pilots, appear to be relatively 
unaffected by the motion used in training. 

To clearly visualize the differences in obtained manual control skills after training with the three different motion 
configurations, Fig. 12 depicts the average frequency responses of the visual and motion channels for each training 
motion group at the end of the training session. These responses were calculated using the asymptotic values iVoo of 
each pilot model parameter in the training session with Eqs. 3 and 4. 

The same observations can be made from Fig. 12 as from Fig. 11. Fig. 12a indicates that the no-motion group 
has the lowest gain at low frequencies, while the roll-only group has the highest gain. The no-motion group also 
starts to generate visual lead at lower frequencies, followed by the reduced-coordinated-roll and the roll-only groups. 
It can be observed that the gain of the motion response is higher for the roll-only group. Finally, it also seems that 
the neuromuscular peak frequency is slightly higher and the neuromuscular damping slightly lower for the roll-only 
motion group. 

IV.D. Crossover Frequencies and Phase Margins 

The open-loop frequency responses were calculated for both the disturbance and target forcing function inputs (Eqs. 5 
and 6). Fig. 13 presents the disturbance and target crossover frequencies and phase margins. The difference between 
training motion conditions is more evident in the phase margin of the target open loop and in the crossover frequency 
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(a) Target open loop. 


(b) Disturbance open loop. 




run 


run 
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Figure 13. Open-loop characteristics. 


of the disturbance open loop. These effects of motion on the disturbance crossover frequency and target phase margin 
have been observed in previous manual control studies . 5 

The disturbance crossover frequency is highest for the roll-only training condition and the lowest for the condition 
in which no motion was present. The roll-only training condition seems to yield the lowest value for the target 
crossover frequency at the end of training. The relative order of the target phase margin between motion groups in the 
training session is different from the order of the disturbance crossover frequency. The roll-only motion condition has 
the highest phase margin, while the coordinated roll condition shows the lowest phase margin. The phase margin of 
the no-motion training condition lies in-between. This relative order is also visible in the disturbance phase margin. 


V. Discussion 

This study investigated the effect of training with different motion configurations on the acquisition and transfer of 
manual control skills in a roll-tracking task. Pilots trained for half a day in one of three motion conditions - no motion, 
roll motion only, or reduced coordinated roll motion - and then transferred to full coordinated roll motion the next 
day. All other task variables, such as the controlled dynamics and forcing functions, were the same in both sessions. 

The experiment had a between-subject design, with at least six pilots assigned to each motion group. However, one 
subject did not finish the experiment, two did not control the task according to standards and the data from two pilots 
did not allow for accurate pilot model parameter estimates. Eliminating these pilots resulted in having only four pilots 
for the no-motion and reduced-coordinated-roll groups. This, in combination with the between-subject design of the 
experiment, resulted in limited significant differences between motion groups in some of the dependent measures. In 
order to increase the significance of some of the observed differences, more pilots might be tested in the future. 

As expected, training without motion results in the poorest performance in the training session, as indicated by the 
higher RMS e values. The tracking performance seems to be the best when pilots are trained with roll-only motion. 
Interestingly, tracking performance of the group who trained with reduced coordinated roll motion was very similar to 
the no-motion group. These differences between motion groups seem to be retained in the transfer session with full 
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coordinated roll motion. The much better performance for the roll-only motion condition may partly be the result of 
the false tilt cues, which provided an increased sensation of motion. In addition, the roll motion gain for the roll-only 
motion condition was twice as large as the gain for the reduced-coordinated-roll motion condition. Similar observation 
can be made for the control activity. Without motion, pilots have a lower control activity in the training session, while 
pilots training with roll motion show higher levels of control activity. Similar trends between no-motion and motion 
conditions are observed in previous research. 3 In the transfer condition, the pilots who trained with roll-only motion 
show a more consistent control intensity compared to the other training conditions. This might be attributed to the fact 
that the simulator roll gain is the same for training and transfer for this group. 

The presence of large false cues in the roll-only condition has different effects on the investigated dependent 
measures. For instance, while performance was the highest when training with this motion condition, the highest VAF 
was achieved for the pilots training with reduced-coordinated-roll motion. One possible explanation is that, without 
turn coordination, the false tilt cue of the simulator induces biodynamic feedthrough, which reduces the linearity of 
pilot control inputs. Further analyzing the VAF, the starting values at the beginning of training seem to be very similar 
for all three motion groups. During training, however, pilots seem to be able to reduce their remnant, allowing for a 
larger portion of the control variance to be explained by the linear pilot model. This effect is reflected by the increasing 
trend of the VAF. This happens especially in the groups who trained with motion, suggesting that motion feedback 
allows for more linear behavior in this task. Although differences are not significant, the relative order of the VAF 
values at the end of the training session is maintained in the transfer session. 

The effects of motion are reflected on the pilot model parameters when comparing the three different training 
motion conditions. Even though not necessarily apparent in Fig. 11, the trend of the parameter evolution is not always 
the same for every pilot. For instance, some pilots who started the experiment with a very high visual gain gradually 
decreased it over the course of training. The opposite trend was adopted for other pilots who started with a low visual 
gain in the same motion group. These large differences between pilots contributed to the large confidence intervals 
for the initial values of the exponential curves. At the end of training, however, pilots tend to approach the same level, 
resulting in smaller confidence intervals for the asymptotic values. 

Having motion feedback increases the visual gain of pilots and replaces the need to obtain lead information from 
visual cues only. Pilots start generating visual lead at higher frequencies (lower lead time constant), since motion cues 
provide the required lead at lower frequencies. Pilot motion gains increase over the course of training, indicating that 
pilots gradually learn to use motion to their advantage. On the other hand, pilots who trained without motion show a 
larger change in motion gain in the transfer session than pilots who trained with motion. The roll-only configuration 
resulted in the highest visual gain, lowest lead time constant and the highest motion gain. This trend is also observed 
in previous research. 

An extra remark can be made about the neuromuscular parameters observed at the end of the training session. 
Fig. 12 suggests that the roll-only motion condition results in a slightly higher neuromuscular frequency and lower 
neuromuscular damping at the end of training. This might also be attributed to possible biodynamic feedthrough, 
resulting from the false tilt cues. 

The highest target forcing function phase margin is achieved with the roll-only training condition, followed by 
the no-motion condition. The reduced-coordinated-roll condition yields the lowest phase margin. Similar trends were 
found in a previous study, 5 where the highest target phase margin was achieved with a roll gain of 1 .0 and a break 
frequency of 0.0 rad/s and the lowest target phase margin for a roll gain of 0.5 and a break frequency of 0.5 rad/s. 
These motion parameters are equivalent to the roll-only and reduced-coordinated-roll motion conditions of the current 
experiment. Event though the trends in the target phase margin are similar, differences were present in the controlled 
dynamics and the lateral motion cueing between the two experiments. This may suggest that roll motion fidelity has a 
bigger impact on the objective results than the fidelity of compensatory lateral motion. 

The experiment contained three different training motion groups. In the roll-only-training-motion condition, the 
roll gain was 1.0 and no lateral motion was present to compensate for the false tilt cues. In the reduced-coordinated- 
roll-motion condition, no false tilt cues were present, but the roll gain was 0.5. The significant differences found 
between these two motion conditions might not only be the result of the false tilt cues, but could also just be caused 
by the difference in roll gain. To eliminate this confound in a future experiment, a roll-only condition should be added 
with a roll gain of 0.5. 

Pilots were asked to leave comments after the end of the training and transfer sessions. Notable comments were 
received by pilots who trained with the roll-only motion condition. They often mentioned that motion sensations were 
comparable to real flight even though large false cues were present. However, after performing the transfer session with 
the full coordinated roll motion, their opinion changed, stating that this motion configuration was even more realistic 
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than the training. This might be caused by the lack of motion-base simulator experience of the general aviation pilots 
who participated in the experiment. 

During the course of the experiment, a drop in performance often indicated the presence of fatigue. Improvements 
in performance were always observed after pilots took a short break. This indicates that fatigue affects the acquisition 
of manual control skills. To optimize the training of manual control skills, future work should also investigate optimal 
training procedures, such as the optimal length of training sessions and the number of training sessions. 

VI. Conclusions 

An experiment was performed on the NASA Ames VMS to investigate the effect of different motion configurations 
on training and transfer of training of manual roll control skills. Of specific interest were the skills needed to control 
unstable roll dynamics of a mid-size transport aircraft close to the stall point. Three different motion configurations 
were used for training: no motion, uncoordinated roll motion, and reduced coordinated roll motion. All pilots trans- 
ferred to full coordinated roll motion. The main difference between the motion configurations was the presence of 
large tilt miscues in the uncoordinated roll case. A novel multimodal pilot model identification technique was success- 
fully applied to characterize how pilots’ use of visual and motion cues changed over the course of training and after 
transfer. Pilots who trained with uncoordinated roll motion with false tilt cues had significantly higher performance 
during training and after transfer. Furthermore, pilot control behavior significantly changed during the two sessions, 
as indicated by increasing visual and motion gains, and decreasing lead time constants. Pilots training without motion 
showed higher learning rates after transfer to the full coordinated roll motion case. The results indicate that motion 
improves the training of manual roll control skills, even when false tilt cues are present. 
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Appendix 

The table below shows a one-way analysis of variance (ANOVA) on the experimental dependent measures for 
each group of pilots. Before the analysis was conducted, checks for outliers, normal distribution, and homogeneity 
of variances were performed. Most of the data contains outliers, which were not removed due to the small pilot 
sample size; however, the majority of the data is normally distributed, as assessed by Shapiro- Wilk’s test of normality 
(p > 0.05). There was homogeneity of variances for most of the data, as assessed by Levene’s Test of Homogeneity, 
where p > 0.05. In the future, more pilots could be tested to provide better consistency in the data. 


Table 3. ANOVA results. 





Training 




Transfer 



Measure 

df 

F 

P 

Sig. 

df 

F 

P 

Sig. 


N 0 

2.0, 11.0 

0.864 

0.448 

- 

2.0, 11.0 

2.073 

0.172 

- 

RMS e 

iVoo 

2.0, 11.0 

4.435 

0.039 

** 

2.0, 11.0 

0.396 

0.683 

- 


A 

2.0, 11.0 

0.324 

0.730 

- 

2.0, 11.0 

0.429 

0.662 

- 


N 0 

2.0, 11.0 

1.835 

0.205 

- 

2.0, 11.0 

0.790 

0.478 

- 

RMSu 

Noo 

2.0, 11.0 

4.706 

0.033 

** 

2.0, 11.0 

0.867 

0.447 

- 


A 

2.0, 11.0 

0.209 

0.815 

- 

2.0, 11.0 

1.834 

0.205 

- 


No 

2.0, 11.0 

3.527 

0.066 

* 

2.0, 11.0 

1.565 

0.252 

- 

K v 

Noa 

2.0, 11.0 

12.485 

0.001 

** 

2.0, 11.0 

1.533 

0.259 

- 


A 

2.0, 11.0 

0.568 

0.582 

- 

2.0, 11.0 

1.227 

0.330 

- 


No 

2.0, 11.0 

18.808 

0.000 

** 

2.0, 11.0 

1.961 

0.187 

- 

T l 

iVoo 

2.0, 11.0 

73.208 

0.000 

** 

2.0, 11.0 

0.287 

0.756 

- 


A 

2.0, 11.0 

6.780 

0.012 

** 

2.0, 11.0 

13.327 

0.001 

** 


N 0 

2.0, 11.0 

10.767 

0.003 

** 

2.0, 11.0 

2.060 

0.174 

- 

T v 

Noa 

2.0, 11.0 

1.501 

0.265 

- 

2.0, 11.0 

1.378 

0.292 

- 


A 

2.0, 11.0 

15.250 

0.001 

** 

2.0, 11.0 

0.195 

0.826 

- 


No 

1.0, 8.0 

1.085 

0.328 

- 

1.0, 8.0 

0.001 

0.971 

- 

Km 

Noa 

1.0, 8.0 

1.932 

0.202 

- 

1.0, 8.0 

0.011 

0.919 

- 


A 

1.0, 8.0 

0.414 

0.538 

- 

1.0, 8.0 

0.025 

0.878 

- 


No 

1.0, 8.0 

4.177 

0.075 

* 

1.0, 8.0 

4.132 

0.077 

* 

T m 

iVoo 

1.0, 8.0 

0.703 

0.426 

- 

1.0, 8.0 

0.511 

0.495 

- 


A 

1.0, 8.0 

0.036 

0.854 

- 

1.0, 8.0 

0.298 

0.600 

- 


No 

2.0, 11.0 

1.426 

0.281 

- 

2.0, 11.0 

0.509 

0.615 

- 

Cnm 

iVoo 

2.0, 11.0 

3.345 

0.073 

* 

2.0, 11.0 

1.741 

0.220 

- 


A 

2.0, 11.0 

3.561 

0.064 

* 

2.0, 11.0 

3.231 

0.079 

* 


No 

2.0, 11.0 

1.354 

0.298 

- 

2.0, 11.0 

0.250 

0.783 

- 

UJnm 

iVoo 

2.0, 11.0 

2.946 

0.095 

* 

2.0, 11.0 

1.034 

0.388 

- 


A 

2.0, 11.0 

1.022 

0.392 

- 

2.0, 11.0 

0.008 

0.992 

- 


N 0 

2.0, 11.0 

0.054 

0.948 

- 

2.0, 11.0 

4.479 

0.038 

** 

VAF 

iVoo 

2.0, 11.0 

3.894 

0.053 

* 

2.0, 11.0 

3.964 

0.051 

* 


A 

2.0, 11.0 

0.654 

0.539 

- 

2.0, 11.0 

0.201 

0.821 

- 


No 

2.0, 11.0 

0.622 

0.555 

- 

2.0, 11.0 

0.944 

0.418 

- 

C,t 

N 0 o 

2.0, 11.0 

10.794 

0.003 

** 

2.0, 11.0 

2.886 

0.098 

* 


A 

2.0, 11.0 

1.080 

0.373 

- 

2.0, 11.0 

1.106 

0.365 

- 


No 

2.0, 11.0 

19.761 

0.000 

** 

2.0, 11.0 

4.185 

0.000 

** 


iVoo 

2.0, 11.0 

40.901 

0.000 

** 

2.0, 11.0 

5.309 

0.024 

** 


A 

2.0, 11.0 

0.768 

0.487 

- 

2.0, 11.0 

1.589 

0.248 

- 


No 

2.0, 11.0 

4.195 

0.044 

** 

2.0, 11.0 

0.685 

0.525 

- 

^ c,d 

Noo 

2.0, 11.0 

6.820 

0.012 

** 

2.0, 11.0 

0.795 

0.476 

- 


A 

2.0, 11.0 

0.616 

0.558 

- 

2.0, 11.0 

0.851 

0.453 

- 


No 

2.0, 11.0 

2.120 

0.166 

- 

2.0, 11.0 

10.410 

0.003 

** 

4 > m,d 

iVoo 

2.0, 11.0 

6.141 

0.016 

** 

2.0, 11.0 

10.303 

0.003 

** 


A 

2.0, 11.0 

0.181 

0.837 

- 

2.0, 11.0 

3.376 

0.072 

* 
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